Welcome back.

And now we will go on with our lecture.

And this part of the lecture is the central one.

Because if you really want to do a project and you have an outline in which steps you

can do this project.

And we will put together the individual ideas we have developed in the last days together

to a whole framework in which you can do whatever application project.

And you have seen that I typically will view the whole design from an architectural viewpoint.

So it's not too astonishing that I have neural network architectures for nonlinear regression

that we speak about nonlinear regression as obviously when you speak with the neural network

stuff.

And then let me make one general comment in advance.

You have seen a similar slide to this before.

This is on the Bayesian model building.

And if you speak about Bayesian model building, then the idea was to say we have to join probability

between models and data and we can split this joint probability in two directions in a conditional

probability.

We can say joint probability can be written as model given the data multiplied by the

probability of the data.

And we can say probability of the data could be generated by the model multiplied by the

probability of the model.

Then you take this and this part and then you come to this formula.

What is the Bayesian model building idea?

You say the probability of a model after we have seen the data is equal to the probability

of some model before you have seen the data.

And this is multiplied by the likelihood in which the data could be generated from such

a model here divided by probability of the data if you have several data points, distributions

of data.

So normally if you look at data analytics lectures, you then will focus completely on

this probability to find a model which explains the data in a good way.

I would like to work out for you that the intellectual thinking about the models, the

model class that you take before you include the data here, that this is also an important

point.

So it's your thinking, it's your insight which is half of the payment for a good project.

Now let's start with it.

The one discussion point I had so often in conferences on forecasting was that people

have argued, now let's start with a simple model, let's do something that's good enough

for paper.

The next year we can work on a more complicated model.

So the question is what is a simple model?

And normally people understand linear models as simple models, but I believe this is wrong.

A simple model must be a model which hasn't any constraints in the model building, which

means for me simplicity is equivalent to universal approximation framework.

Obviously neural networks are an example of a universal approximation framework, but there

are others.

For instance, you can take any Taylor expansion or whatever expansion you want to take, but

you have to start with universal approximation framework.

The other understanding of simplicity is linear models are so easy to do, which means on the

method side they are so easy, so therefore they are simple.

But it's wrong if you want to get an understanding of the world.